Bayesian Monte Carlo for the Global Optimization of Expensive Functions

In the last decades enormous advances have been made possible for modelling complex (physical) systems by mathematical equations and computer algorithms. To deal with very long running times of such models a promising approach has been to replace them by stochastic approximations based on a few model evaluations. In this paper we focus on the often occuring case that the system mod­ elled has two types of inputs x = ( x c , x e) with x c representing control variables and x e representing environmental variables. Typ­ ically, x c needs to be optimised, whereas x e are uncontrollable but are assumed to adhere to some distribution. In this paper we use a Bayesian approach to address this problem: we specify a prior distri­ bution on the underlying function using a Gaussian process and use Bayesian Monte Carlo to obtain the objective function by integrating out environmental variables. Furthermore, we empirically evaluate several active learning criteria that were developed for the determin­ istic case (i.e., no environmental variables) and show that the ALC criterion appears significantly better than expected improvement and random selection. 1 I n t r o d u c t i o n Optimisation of expensive functions is one of the core problems in many of the most challenging problems in computing. Mathemati­ cal computer models are frequently used to explore the design space to reduce the need for expensive hardware prototypes, but are often hampered by very long running times. Much emphasis has therefore been on optimising a model using as few function evaluations as pos­ sible. A very promising approach has been to develop a stochastic approximation of the expensive function to optimise a surrogate model and use that approximation as replacement in optimisation and to determine the next best function value to evaluate according to some criteria in model fitting. This approach is well known as re­ sponse surface m odelling [11, 9]. In this paper we consider a situation often observed in practice in which there are two types of input variables: x = (x c, x e) with x c a set of control variables and x e a set of environm ental variables. The control variables are the variables that we can control whereas the en­ vironmental variables are assumed to have values governed by some distribution that we cannot manipulate. For example, in [3, 2] a hip prosthesis is designed where the control variables specify its shape and the environmental variables account for the variability in patient population like bone density and activity. In [29] a VLSI circuit is designed where the control variables are the widths of six transistors and the environmental variables are qualitative indicators. In [12] a compressor blade design is improved where the control variables 1 Radboud University Nijmegen, Institute for Computing and Information Sciences, the Netherlands, email: [email protected] specify the geometry of the blade and the environmental variables are manufacturing variations in chord, camber, and thickness. In this article we focus on optimising a real-valued objective func­ tion that only depends on the control variables, but its value for each setting of the control variables is the mean over the distribution of the environmental variables. Hence, we seek to optimise the control variables in order to obtain the best average response of the objective function over the distribution of environmental variables x* = a rg m a x ¿ (x c) = argm ax / f ( x c , x e) p ( x e) d x e (1)